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ABSTBACT 

Military aircraft missions are all multidimensional in 
nature. This means that every mission can be divided into 
usually one overall goal or purpose (i.e.^ destroy the 
target/ deliver the supplies^ rescue the survivors, etc.), 
with several subgoals (safety, minimize susceptibility, 
timeliness, etc.). Since missions are multidimensional, the 
operator effort in the form of mental and physical action 
(performance) becomes multidimensional. The multidimentional 
nature of skilled aircrew performance, in turn, requires 
that several criteria, all of which are relevant for a 
particular activity, be defined and used [Ref. 1]. 

The unique situation of an aircrew flying an aircraft 
for a specific mission and the necessary determination of 
subcriteria for evaluating accomplishment of that mission 
requires further research of an analytical and empirical 
nature. The relationship among altitute, airspreed, 
operator activity, and the hundreds of other system vari- 
ables that comprise the total system must be compared to 
mission success in quantifiable terms. 

This study is an effort to improve acquisition of 
training performance information in affortable ways on 
behalf of the Hellenic Air Force (HAF). Thus, it is divided 
in: (1) The principles of human performance, (2) Definition 
of the criteria and their measurement, (3) Systematic defi- 
nition of performance measure appropriate to combat- training 
needs, (4) Definition of a cost effective measurement system 
usable in combat-crew training environments to acquire and 
process needed training information. 

The method that was used was a search of the available 
materials found in the NPS library. Most of the data found 
was based on DSAF studies. 
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I. INTBODUCTION 



The Hellenic Air Force (HAF) is interested in optimizing 
its combat readiness, and conseguently the combat readiness 
of its combat pilots. Greece has common borders with 
hostile countries and is constantly threatened with possible 
invasion. There are continuous violations of the aerial and 
sea space of Greece by neighboring countries* airplanes and 
ships respectively. Se (the Greeks) must offset our 
numerical inferiority with technology expressed in higher 
skilled combat pilots. It is imperative to increase to the 
utmost the combat readiness and effectiveness of the HAF. 
In this way we shall be a strong power and our enemies will 
never try to invade our country. Thus, we shall remain in 
our purpose, that is, united and uninvaded country and most 
important, because of our power we* 11 keep the peace. 

Today the need for combat effectiveness is more pressing 
than ever before. We live in a period of rapid and 
continuous change in technology and conseguently in our 
tactics. The new generation of combat aircraft require more 
training because of the complexity of the new systems. 
Because of these technological changes it is required that 
new changes in air tactics must be established. Thus, 
because of the above continuous changes, it is required that 
new criteria for combat readiness of combat pilots must be 
established (created). Of course, HAF has established 
criteria for its combat pilots as a result of the entry of 
the present generation of combat aircraft. However, it may 
be necessary to see if there are any critical points that 
need to be corrected or changed. For these reasons this 
study has been conducted in which the US Air Force was used 
as a model and a later study will be necessary to adapt 
these models to the Greek reality. 
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this study will define combat readiness. 



the 



Thus, 

criteria and their considerations and human performance, 
relation to the combat readiness of combat pilots. 



in 



A. COMBAT READINESS 

But what is combat readiness? In order to answer this 
question, we first must define the word readiness, secondly 
combat readiness, and third readiness categories. 

Readiness means the quality of being ready to act, 
respond, comply, etc; Also readiness is a state or fact of 
being ready, or prepared; as to have everything in readiness 
for a sudden departure [Ref. 2]. A crew or crew member is 
comb at ready when it is certified as ready for combat 
[Ref. 3]. 

1 . Readiness Ca tegories : 

Combat readiness is expressed with the following 
standard categories and meanings [Ref. 4]. 

a. Fully Combat Ready. 

A unit fully capable of performing the 

mission(s) for which it is organized or designed. 

b. Subs tant ially Combat Ready. 

A unit capable of performing the mission (s) for 
which it is organized or designed, but having minor defi- 
ciencies which could reduce its effectiveness or its ability 
to conduct sustained operations. 

c. Marginally Combat Ready. 

A unit with major deficiencies of such magnitude 
as to severely limit its capability to perform the 
mission (s) for which it is organized or designed, but 
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capable of conducting limited operations for a limited 
period. 

d. Not Combat Ready. 

A unit not capable of perfoming the mission (s) 
for which it is organized or designed. 

Definitions of readiness and combat readiness 
were given above. A classif ication of combat readiness into 
four categories was made in an effort to specialize and 
highlight it. In order to define and understand combat 

readiness better we have to define criteria and their 
considerations, that is to say, definition and purpose of 
criteria, the categories of criteria and characteristics of 
good criteria. 

E. CRITERIA AND CONSIDERATIONS 

1 • D efini t ion and Purp ose of Cri teria 

Criteria are standards, rules, or tests by which 
measures of system behavior are evaluated in terms of 
success or failure, or to some degree of failure. The 
purpose of human performance criteria is to provide 
standards or baselines for evaluating the success or 
failure, goodness or badness, or usefulness of human 

performance [Ref. 5] and [Ref. 6]. 

Criteria should not only define the unique manner in 
which the operator should perform a task, but should define 
the performance objectives of the entire man-machine system 
[Ref. 7 ] and [Ref. 8 ]. 

2 • Categor ies of Crite ria 

The classif ication of criteria can be accomplished 
from a measurement standpoint; beginning with the smallest 
known entity and ending with the "ultimate” quantity that 
may exist. Several categories identified are listed below: 
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(1) Parametric referent or standard of performance 
which is sought to be met by the operator or system. 
Example: maintain 1000 feet of altitude [Ref. 7] and 
[Ref. 9]. 

(2) Parametric limit about the parametric standard 
within which the operator or system is required, or 
seeks, to remain. Example; maintain plus or minus 
100 feet while at 1000 feet altitude [Ref- 7] and 
[Ref. 9]. 

(3) System component criteria which distinguishes 
the relationship between system components and system 
output. Example: " least effort " measured from 
the pilot in relation to maintaining altitude 
[Ref. 10]. 

(4) Test criterion used to evaluate overall human 
ability, usually expressed as a single overall 
measure- Example: subjective judgement of instructor 
for a student as to ’’pass” or "fail”. 

(5) Ultimate criteria are multidimensional in nature 
and represent the complete desired end result of a 
system. This category of criteria is impossible to 
quantify due to the multidimensional nature of the 
system’s purpose, and hence, is a theoretical entity 
that must be approximated- Example: Any aircraft’s 
mission [Ref- 1] and [Ref- 11]- 

It may be noted that all five categories of criteria 
can be either quantified or approximated in some manner, 
with decreasing accuracy as the ultimate criteria level is 
reached. 

Obtaining direct measures of the ultimate criteria 
for a complex system is seldom feasible. This is particu- 
larly true in military systems where such criteria would be 
expressed in terms of combat eff ectiveness or effectiveness 
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in preventing a potential aggressor from starting a conflict 
[Ref. 1]. 

Therefore, it becomes apparent that we must select 
intermmediate criteria (types (1) through (U) above) in 
evaluating skilled operator behavior. 

3. Characteristics of Good Cr iteri a 

Using actual criteria as approximations of the ulti- 
mate criteria can be accomplished by several methods that 
will be presented in the following paragraphs . 

Although there is no certain method that will lead 
to the specification of good criteria, there are seme 
considerations that can be taken into account which are 
discussed below: 

(1) A good criterion is both reliable and relevant 
C Ref. 1] and [ Ref. 10]. 

(2) Criteria must be comprehensive in that the utility of 
the individual being evaluated is unambiguously reflected. 

(3) Criteria should possess selectivity and have ready 
applicability [Ref. 10]. 

Finally, since readiness and criteria are related to 
human performance, it is necessary to define human 
performance. 

C. BOHAN PERFOfiflAHCE 

Much of the data used in human factors consist of 
measures of some aspect (s) of human performance. Human 
performance in the context of systems often boils down to 
consideration of how fast people can perform their functions 
and how accurately they can perform them. How well functions 
are performed leads directly to considerations of human 
errors [Ref. 12]. 
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D. THE PROBLEM 

Thus, the problem is the necessity of optimi2ing the 
combat readiness of HAF's combat pilots, because of the 
threatened invasion by neighboring countries. So, we have 
to identify the necessary variables which will assist the 
planner (commander) to be certain that his combat pilots 

will meet the desired level of combat effectiveness in terms 

% 

of pilot capabilities. 

It can be suggested that for HAF the first requirement 
is to continuously optimize combat readiness. To accomplish 
this objective it is essential that combat pilots be at an 
acceptable level of combat readiness. Determination of 
combat readiness requires definition of concept, specifica- 
tion of criteria and a definition of human performance. 

In the chapters to follow an attempt will be made to 
define and discuss the following; (a) the principles of 
human performance and (b) the criteria and their measure- 
ment. Finally, chapters IV and V will examine combat ready 
crew performance and derivation of combat-crew performance 
measurement techniques. 
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II- PRIN CIPLE S OF MHAI PERFOHMANCE 



A. HDHAN BERFOBHANCF BEQUIREMENTS 

W€ can best determine performance requirements only 
after we have a good set of allocated^ functions. In the 
absence of function allocation we will have a difficulty 
identifying and dealing effectively with human performance 
requirements because we do not know what functions people 
will be expected to perform. 

Once we have established a human performance requirement 
we should develop a way of measuring it. If the requirement 
relates to accuracy, then there must be a meaningful way to 
measure errors. If the requirement concerns manual 
processing time, there must be a meaningful way to measure 
it. For example, time per customer contact, number of items 
produced per hour, and average keystrokes per day. 

Usually the problem is deciding what measure gives the 
best indication of performance. It has been suggested that 
with a little imagination, any human performance can be 
meaningfully measured [Ref. 13]. As a minimum, human 
performance requirements should include statements 
concerning errors, manual processing time, training time 
necessary to ensure the minimum skills, and job 
satisfaction. If we do not clearly state the requirements 
from the beginning we cannot expect human performance 
considerations to be taken seriously. And when the system 
is operational, people will be left to perform as best they 
can without adequate provision for ensuring an acceptable 
level of human performance [Ref. 14]. 

Other human performance requirements could be associated 
with training time (e.g., total time to train clerical 
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persoDDel to perform the basic activities should not exceed 
three weeks). Another human performance requirement could 
relate to job satisfaction (e.g., after performing an 
activity for six months employees should respond in a posi- 
tive way to their work, as measured by a questionnaire) . 

The identification of human perfomance requirements is a 
prerequisite step for a system’s development process. One 
of the steps of a system’s development process is the task 
analysis. Task analysis is intended to match the work to be 
done with the kinds of people who will do it. The process 
has four main parts: 

(1) Determination of a system structure that gives the 
designer an overall view or objective for the analysis. 

(2) Identification of tasks has essentially nine primary 
considerations associated with it. These include 
determining existing knowledge and skills, deriving skill 
level categories, identifying outputs and inputs, deriving 
lower level activities, ensuring that activities are mutu- 
ally exclusive and exhaustive, and matching active 
complexities with previously determined skill levels. The 
process may be repeated any number of times until each 
activity is assigned a single skill level. Secondary 
considerations in the identification of tasks include 
meeting the system structure objective just discussed, as 
well as meeting a full-advantage objective. A designer 
attempts to develop tasks that will ultimately take full 
advantage of the user work force. This is difficult to 
quantify, but during the analysis process, most designers 
gain a feel for what is meant by taking full advantage of 
the skills available in their user population and this 
should be reflected as the tasks are being identified. 

(3) Description of each task and organization of all tasks 
into a flowchart that will accomodate the variety of 
different transactions a new system must accomplish. 
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(4) Identification of work modules that synthesizes the 
tasks previously identified into manageable modules of 
work. A human wo rk modu le is a set of tasks that a user 
accomplishes as a part or all of his or her job [Ref. 14]. 
It is a basic unit work. Usually, one or more work modules 
are combined to form a job. 

Having systematically derived work modules assists in 
the design of interfaces, and the preparation of facilitator 
materials, such as instructions, performance aids and 
training. 

B. FACILITATING HOHAH PERFORMANCE 

Once good work modules are developed, many types of 
materials can be developed to help ensure an acceptable 
level of human performance. All of these materials are 
based on the task analysis results and are usually prepared 
to support the work module instructions. 

Once the designer identifies tasks and determines work 
modules, he must identify the specific skills and knowledge 
required for each work module. This amounts to stating his 
assumptions of the precise qualifications of the person to 
perform the work module. 

As the work modules are designed, the designer should 
have a set of user character istics in mind. The design 
should clearly envision the person performing the work. 
When this information is written down, it is called a 
Statement of Minimum Qualifications or SMQ [Ref. 14]. 
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The SMQ is a detailed description of minimum acceptable 
gualif ications in terms of skills and knowledge required to 
efficiently and effectively perform the work outlined by the 
work module. 

Once the designer has in mind the potential user and has 
designed the work modules, then he can proceed with 
developing instructions, performance aids and training. 

It is vitally important that human performance 
considerations begin at least by the time functions are 
being allocated. Having a good set of human performance 
reguirements is also critical. A systematic and detailed 
analysis of the tasks to be performed is also very 
important. 

If design considerations are well done, then a strong 
foundation is established for developing interfaces and 
facilitators, including Human/Computer dialog structures, 
instructions, performance aids, training materials. 

In this chapter the principles of human 
performance, namely (a) Human Performance reguirements 
and (b) Facilitating human performance have been discussed. 

In order to measure one person's attitude, aptitude and 
performance we may use criteria. Thus, in the next chapter 
we shall expand on the concept of criteria and their 
measurement. 
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III. CRIIERIA AND THE^ MASORMMT 



As suggested earlier, a criterion is a standard which 
may he used to evaluate a person's attitude, aptitude, and 
performance. Since personnel selection and training are 
important factors for HAF to optimize combat readiness of 
its combat pilots, we will be concerned with the use of 
criteria for personnel selection and training purposes. 

Selection criteria are described by the degree of corre- 
lation between selection test scores and performance 
measures (in real-world situations) , which represent the 
degree of a pilot's success in performing his job. 

Training criteria refer to measures utilized in evalu- 
ating the effectiveness of a training program (i.e.,the 
measures that express the degree to which the attainment of 
the behaviorial objectives of the training program have been 
met) [Ref. 15]. 

A. OSE CF CRITERIA IN PERSONNEL SELECTION AND TRAINING 

Personnel selection serves to predict a person's 
suitability for a job, whereas the purpose of personnel 
training is to derive a predetermined work standard, or 
other criterion, in the shortest possible training time. 
Both purposes can be achieved by analyzing and quantifying 
the content and skills associated with the job. The valida- 
tion of each process can be evaluated as follows : 

1 . The validity of a personnel selection system can be 
evaluated by the degree of the relationship, or correlation, 
between test scores against the criteria measures. 

2. The validity of a training method can be evaluated by 
comparing a variety of different training methods with a set 
of the same criteria for all cases. 
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The validity of selection and traininj programs alike is 
affected by the following factors : 

1. The degree to which personnel selection tests or 
personnel training procedures simulate real-world criteria. 

2. The validity of personnel selection tests and 
training under study. 

3. Validity of the criteria considered, true and not 
merely correlational. 

4. Eelevance of the criteria considered for (a) the job 
studied, and (b) the test batteries adopted for selection or 
training purposes. 

5. The stage (initial, ultimate, or rate of learning) at 
which criteria are measured may affect validities signifi- 
cantly. Typically, criteria measures at the early stages of 
the acquisition of skill exhibit higher validity coeffi- 
cients than those we find when the measures are correlated 
with terminal performance [Ref. 16]. 

6. Because job performance is multidimensional, taking 
only one criterion as an index of job performance may result 
in artificially high or low validity coef f icients. 
Therefore, a multidimensional approach is essential in the 
measurement of job performance and criteria utilization in 
personnel selection and training. 

7. Evidence indicates that criterion and test measures 
cire typically nonlinear [Ref. 17] [Ref. 18] and [Ref. 19]. 
Since, moreover, assuming their linearity increases the 
error variance of the validity coefficient, the validity of 
the assumption must be tested before linearity is assumed. 

8. Fluctuations in an individual’s physiological and 
psychological health condition, need of achievement motiva- 
tion, and so on, affect the validity coefficient of 
personnel selection programs as well as training programs. 
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In summary, it is hardly possible to overstress the 
importance of a sound criterion measure against which 
personnel selection and training programs can be evaluated 
effectively. In order to achieve this objective, emphasis 
must be placed at the outset on the development of such 
measures. Without sound criteria the true validity of 

personnel selection and training cannot be assessed 
properly. 

B. TIPES CF CEITEEIA. 

Criteria for the purposes of personel selection and 
training lie on a continuum scale, which has two interactive 
dimensions : objectivity-subjectivity and employee-employer 

satisfaction. Some criteria are entirely subjective; others 
are entirely objective. Most, however, involve a mixture of 
the two. In other words we distinguish two types of 

criteria; (1) Those describing the satisfaction of the 
empoyee*s needs and (2) those describing the satisfaction of 
empoyers in their empcyees. 

The ideal criterion, although rarely found in industry, 
is the one that reflects the satisfaction of employee's 
needs as well as that of empoyers*. A criterion like this 
is used more frequently for personnel guidance purposes than 
in personnel selection and training [Eef. 20]. 

It is essential to aim at using the most appropriate 
criterion for a specific job, rather than the most 
convenient criterion, but it is not always feasible to 
assess the former or to obtain data on it. Thus, the 

analyst is frequently faced with the practical issue of 
using the most appropriate criterion which data are 

available or can easily be collected. 

The reason for using a criterion varies since occupa- 
tional jobs are different; consequently, a variety of 
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criterion measures have been developed and used during the 
last century, each vith its characteristic weaknesses and 
strengths. However, any criterion could be the most appro- 
priate for a specific job, depending on circumstances and 
purpose. The majority of the criteria relevant to today’s 
occupational circumstances and competence are listed in 
Table 1. • 

For seme of the criteria mentioned in Table 1, objective 
data can be gathered (e. g., length of employee’s service), 
others require subjective assessment (e.g. , rating or 
ranking by supervisor, subordinates, members of equal 
status, and the employee himself) . For some criteria (e. g. , 
quality of output) both subjective and objective techniques 
can be utilized. Many of these can be related to training 
progress at the following stages; (a) immediate, at the 
beginning of training; (b) intermediate, at the end of 
training; (c) ultimate, in real-world operations, after 
completion of training. 

C. CEITZEIA HEASOEES 

There are basically three types of methods by means of 
which criteria data can be collected : 

1. Eating or ranking. 

2. Counting (e.g., the number of items produced, the 
number of accidents, the duration of service) . 

3. Establishing work standards [Ref. 21]. 

In this chapter we have seen the criteria and their 
measurement and specifically (a) Use of criteria in 
personnel selection and training, (b) Types of criteria and 
(c) Criteria measures. Next we shall introduce the combat 
ready crew performance measurement. 
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TABLE 1 

Tytes of Criteria Utilized for Personnel Selection S 

Training 



I. Criteria reflecting employees* competence 



A. Quality of 



T. 

2 . 



uality 

poiled 



3. 



4. 



5. 



6 . 

*7. 

* 8 . 

* 9 . 






D J- e 

Mis 



performance 
of output 
work: 



Ji 



B. 



(ii) to employees 



II. 



1 . 

2 . 

3. 



4- 



III, 

1 . 

2 . 



amount 
cost 

Accidents; 

J a) number 

b) cost (financial and human) 
reakages (tools, etc.) ; 
number 
cost 

istakes in operation; 
a) number 
b) cost 

ariability in performance 
Rate of advancement 
Standard trade examinations 
Training; 

(a) cost 

(i) to employer, 

(b) duration 
Quantity of performance 
1. Quantity of output 

Earnings on a commission basis 
Earned bonus 
Peak performance 
Lowest performance 
Rate of advancement 
Standard trade examinations 
Training ; 

(a) cost 

(i) to employer, (ii) to employees 
(b) duration 

Criteria reflecting employees* circumstances 
Length of service 
Labor turnover 

Lateness (categorized according to how late) 
(a) numbers 
(b) cost 
Absenteeism ; 

(a) certified 

(i) number, (ii) cost 
(b) uncertified 

(i) number, (ii) cost 

Criteria reflecting employee*s satisfaction 
from job 

Rating of employee* s liking for his present 
Rating of employee* s satisfaction with 
job content and desire for job enlargement 
or job simplification 



2 , 

3. 

4. 

5. 

* 6 , 

*7. 

* 8 . 



job 



* Starred items may reflect quality and/or quantity of 
performance. Since their relationship to performance 
IS indirect, they are less valuable and important 
than the others. 
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IV. INTRODOCTION TO COMBAT READJ ^ES PERFORMANCE 

HEASOREMENT 



Recently, the evaluation of aircrew proficiency in 
skills associated with advanced flying training was prima- 
rily based upon the subjective judgement of instructor 
pilots- In addition these evaluations are supplemented by 
the mere or less objective scores of gunnery and bombing. In 
certain areas, such as air combat maneuvering and 
combat- readiness determination the evaluations are totally 
objective. 

Although, efforts have been made to measure objectively 
the behavioral skills in the operational and/or crew 
training setting in an economical way, these efforts have 
not yet produced any positive results. 

A great percentage of the HAF*s budget is absorbed by 
the aircrew training costs, therefore, this area has become 
of fiscal concern- Due to the projected increase in costs 
of operating the newer weapons systems over the present 
generation of combat aircraft the HAF*s interest in the 
problem has intensified in recent years. Studies to minimize 
costs have shown that this can be achieved by reducing the 
training flying hours and transfer them (when/where this is 
possible) to lower cost devices, e-g. simulators. 

The HAF is trying to adapt modern Systems Approach to 
Training techniques to aircrew training programs. Existent 
in the Systems Approach to Training concept is student 
advancement on individual proficiency rather than course 
length. The traditional subjective methods of evaluation may 
be proved insufficient for the more complex aircrew training 
ob jectives- 
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A. OBJECTIVES 



Necessity of improving training performance information 
of HAF’s pilots, directed this study to the following 
objective based on experiments and studies of USAF 

[Ref. 22] : 

Goal: Systematic definition of performance measures 

appropriate to combat-crew training needs. 

Performance measures will include formal statements of 
methods of measuring flight crew performance used 
during and at the end of combat-crew training, and, new 
measures meaningful to combat-crew training and useful 
as tools for training research. 

This study thus is an effort to describe usable measure- 
ment tools for utilization in combat-crew training 
research. 

1 . Instruction al System Devel opment . 

Research studies directed toward performance 
measurement in comhat-crew training are highly relevant 
today in view of USAF policy to employ a systems approach to 
flying training problems. The model for Instructional 
System Development (cf.. Dept. of Air Force, 1970) contains 
the following basic steps: 

1. Analyze system requirements. 

2. Define education or training requirements. 

3. Develop objectives and tests. 

4. Plan, develop, and validate instruction. 

5- Conduct and evaluate instruction. 

In support cf instructional system development, 
measures and a measurement system are necessary to: 
(1) perform analyses of systems in their operational 
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environments, (2) establish quantitative instructional 
standards, (3) provide an index of achievement for each 
behavioral objective, and (4) evaluate alternative 
instructional content, approaches, and training devices. 

In particular, instructional system development 
requires that performance standards are identified so that 
the most efficient approach is used to train the needed 
skills and knowledge to the desired level of performance. 
Such performance standards imply performance measurement for 
both the determination of desirable approaches to training 



and fcr testing student 


performance. 








All these events mentioned 


above 


should 


be 


considered positive 


and therefore 


this 


model 


for 


instructional system development could 


also 


be used 


and 


adapted by HAF. 










2. A Measurement 


System for 


the 


Operational 



E nvironment . 

It will be necessary to develop measurement tools 
that would be usable in the operational environment of HA? 
under the constraints that such an environment implies. 
Within the context of this requirement, it is essential that 
an attempt be made to establish a list of parameters to be 
sensed, and the pcint-of- view taken that the parameters 
should be derived from that information that the operational 
training personnel consider to be meaningful and 
significant. 

3. An Automated Measurement System. 

An ancillary objective is to develop a measurement 
system that will relieve the instructor pilot, to a maximum 
extent, from the requirement of having to record a great 
deal of information manually on the basis that such activity 
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degrades his ability to competently instruct his student. 
This does not imply that such a measurement system is an 
attempt at automated evaluation. The measurement system 
should certainly include means for transforming and 
analyzing performance information, but ultimately evaluation 
and decision for training control is a human function- 

Thus, HAF would improve and optimize the quality of 
training of its combat pilots by adapting an instructional 
system development, a measurement system for the operational 
environment and finally an automated measurement system. 



B. HEASDBEMENT BASIE ON COMBAT-CREW TRAINING INFORMATION 

NIEDS 

The strategy employed by the DSAF was to design a 
measurement system that could acquire that data identified 
as meaningful by training management and instructor 

personnel. Data were gathered from sites across a broad 
spectrum of combat-crew training programs of USAF. 

The combat-crew training sites from which data were 
taken are listed in Table 2. The aircraft sample included 
heavy (inter- and intra- theatre cargo/transport, and 

bomber) and high-performance aircraft (one- and twc-man 
interceptor and tactical fighter) . An attempt was made to 
(1) consider measurement in the context of combat-crew 
training, (2) assess measurement already included as well as 
identify potential measurement indicated by combat-crew 
training personnel, and (3) assess the constraints placed by 
the environment on feasible, usable measurement systems. 

From the six types of aircraft listed in Table 2 the HAF 
pilots fly the three cf them, that is F-4E, C-130E and A-7D. 
Thus, the OSAF*s attempt and data found are of great 
interest for HAF, in order to improve training performance 
information for its ccmbat-cre ws. 
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TABLE 2 

Data Collection Sites 



Place 

Castle AFB 
Altus AFB 
Dyess AFB 
Davis-Monthan AFB 
Tyndall AFB 
Luke AFB 
♦George AFB 
♦Norton AFB 
♦♦Nellis AFB 



Aircra ft 

B-52 F, G S H 

C-141A 

C-130E 

F-4 C, D S E 

F-106 A S B 

A-7D 

F-4E 

C-14 1A 

F-4E 



♦ Special emphasis on crew-performance measurement 
♦♦Special emphasis on air-combat maneuvers 
performance measurement. 
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C. COHBAT-CEEW PEBFOBMANCE MEASOBEMENT 

For the six aircraft mentioned in Table 2, a common 
basis for measurement was established. This way it could be 
possible to apply a more or less modular approach. Allowance 
must be made for special aircraft characteristics ; for 
example, the F-106 has no flaps and the B-52 has a quite 
complicated flap retraction routine compared to other 
aircraft. Measurement was treated for each of the following 
maneuvers: 



(1) Takeoff and Climb 

(2) Pattern, Land and Go- Around 

(3) Instruments 

(4) Formation 

(5) Air-Air Intercept 
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(6) Air Combat Maneuvers 

(7) Air Refueling 

(8) Ground Attack 

(9) Air Drop and Air Drop Formation 

(10) Radar Navigation and Bombing. 

Pro totype measurement . For each of above maneuvers, the 
data collected from the sites of Table 2 were compiled into 
the summary form shown in Figure 4. 1 Since the blanks in the 
summary form of Figure 4. 1 indicate needed information, and 
conseguently items for measurement development, these forms 
were termed Prototype Measurement since they form a model 
after which measurement could be patterned. Details and 
explanations of a prototype measurement is presented in 
Appendix A. 

Me as u reme nt Specifications . The parameters which must 
be sensed to permit measurement are not immediately evident 
from the information requirements (Prototype Measurement) , 
since the measure specifies the output of a computation, and 
the computation itself must be known before the inputs to 
the computation (the parameters) can be determined. 

For example. Figure 4. 1 indicates that a measure of 
centerline deviations is desired during the takeoff roll. It 
is clear that the distance between the aicraft position on 
the runway and the runway centerline is a parameter needed 
for measurement. The desired measurement might be simply the 
average difference, or, conceivably, might involve the rela- 
tionship between centerline deviation and heading (or 
lateral-G, or brake application) and thereby indicate the 
need for other parameters to execute the calculations for 
measurement. Further, the measurement calculations must be 
made (as indicated in Figure 4.1 ) from the application of 

takeoff power until rotation, implying the need for other 
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1 



TAKEOF F S CLIMB* 



CONDITIONS; 

TSross Wt: Wing: / _ Eunway: / 

Temp.; Alt. Set.; PTeld Elev. Form Pos. ; 



TA KEOFF EOLL; (To powe r until rotation) 

"Power S^:_ _ Tenterline Pev. : Min, Max, Av. 

Eeject Speed; Computed Heading;Min, Max, Av. 

Time: Dist;_ Bank; s Max, L Max 



EOTATION; ( Nose g ear off until pitch att. esta bl ishe d) 

Poi.Tpeed: "SlaP. Trim; 

Pitch; Eate ; Bank;_ 

Final; Centerline Pev. ; 

Overshoot; Heading; 



LI FTOFF ; (Pos. Vert. Vel.) 

"Airspeed; “Pitch; Bank; Hdg: 

Vert. Vel.~Xfter: Sec.; 



GEAE-UP; (Handle up until gear-up S locked) 
“G^Tp Speed!" _ VTTTTnitTTII. T.VTITnal; 
Pitch; Bank; Hdg; 



FLAPS UP; (Start up to full up) ** 

TfliT 52 Only 

Pitch: Bank; Hdg: Start 



A/S (IPIT) TFINAL 

VV (INIT)^ (FINAL 

ALT (INITf (FINAL 



1st Pos 
2nd Pos 
Full 



IAS 

X 

X 

X 

X 



PITCH 

X 

X 

X 

X 



ALT VV TEIM 
XX X 

XX X 

XX X 

XX X 



CL IMB & MVEL ^F: (Defends on Flig ht Plan) 



Accelerate 
Climb A/S 

Climb MACH 
Level-off 




INIT FINAL 

£11 AZS MACH HDG ALT ALT 

XX X X X X 



X X 

X X 



X X X X 

X X X X 



Alt-105? VV) 

to Cruise) x x x x x x 



PITCH TRIM 
X x 

X X 

X X 

X X 



mandatory communication S instances where A/C 
s are exceeded. 

**F-106 has no flaps 



j 



* Also- 
limir 



Figure 4.1 Example of Prototype Measurement. 
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parameters to indicate when the measurement interval starts 
and stops. 

Since the pilots of HAF fly at most the same aircraft' 
as USAF pilots^ the data gathered from these USAF tases are 
of great interest for HAF. Of course^ of greatest interest 
are the data of the three similar aircraft (F-4E, A-7D/ and 
C-130E)/ but the data related to the other aircraft should 
also te of great interest. These may be applied to ether 
aircraft that pilots of HAF fly (e.g. F-1C Mirage^ or 
F-5A/B ) , or aircraft that HAF will be procured in the near 
future. 



'The aircraft that the HAF*s 
to the DSAF aircraft bases from 
are: F-4E, A-7D, and C-130E. 



pilots fly and are similar 
where data were gathered 
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debivation of ccm batches peefommce measurement. 



One cf the principal products of the USAF study was the 
definition of perforoiance measures appropriate to combat- 
crew training needs. That study was based on interviews with 
instructors and training management and measure'ment defini- 
tions were evolved in the following steps : 

(1) The varied requirements posed by the six aircraft 
and missions were consolidated into a common framework which 
permitted isolation of measurement modules (measurement 
commonality) ; 

(2) Discussions of operational training information 
needs were formalized to indicate in a checklist fashion the 
measurement development needed (prototype measurement) ; 



(3) Measurement parameters and (4) 
cations were produced together, but are 
for hardware and software implications. 



measurement specifi- 
presented separately 
respectively ; 



(5) Measurement and analysis for crew communications 
recording were examined to provide means of examining crew 
interaction and individual contributions to total man- 
machine system performance. 

A. measobehent commonality 

Common measurement was executed to permit the design of 
one simple and practical measurement system, and eliminate 
the need for a totally unique measurement system for each 
aircraft. This measurement system could apply in HAF. 
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1 . Com mon T rain ing Phases . 

In the DSAF program the first step was to compare 
similar training phases for each aircraft, in order to esti- 
mate the commonality measurement requirements, as it is 
shown in Table 3. In cases where X-Y^ data are required for 
measurement it was found that these data could be obtained 

by using expensive equipment (such as a multiple-target 

% 

tracking radar) or with difficult-to- process recordings, 
while equivalent results could be obtained with video/photo 
sensors. As we can see in Table 3, almost all cases of X-Y 
data requirements can be met by using video/photo recording. 
Those cases where the X-Y data cannot be obtained with 
video/photo sensors are: (1) lateral drift across the 

runway during transition, (2) relative position of aircraft 
during intercept prior to lockon, (3) enroute cross-track 
error during airdrop, (4) inflight ranging (out of sight) 
during formation, and (5) space paths of multiple aircraft 
during air combat mansuvers- 

Although, phcto-sensors are widely used by the HAF, 
video is not. So, the use of video films would be recom- 
mended because video is a modern technology equipment with 
many advantages (e.g., film does not need development and 
therefore can be used immediately after flying during 
debriefing) . 

Not all maneuvers were taught at all sites from 
which data was taken. For example, the operational C-130 
squadron did not explicity train transition maneuvers; 
nevertheless, competent information was obtained for 
measurement. Since all combat maneuvers were not taught at 
the combat-crew training squadrons, they had gaps in data 



X-Y data are not in the form of a tabulation of X- 
and Y 7 values, but they are position information such as the 
relationship between tanker and refueling aircraft from the 
tanker lights. 
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TABLE 3 

Reguirements For X-I Data 



T 



Fhase 



Parame ter 



Obtainable 

wlfl 

ViHeo /Ph oto ? 



1 

2 

■? 

4 

t; 



6 

7 

8 
9 

10 
1 1 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 



Transition Ground Track RA* 

Centerline Dev RA 

Lat. Drift No 

Threshold RA 

Dist. Down RNWY RA 

Spacing RA 

Intercept TgT. Azimuth No 

(Prior TO Lockon) TGT, Elevation No 

TGT, Range No 

TGT, Range Rate No 

TGT, Aspect Angle No 

Air Refueling Tanker Range RA 

Tanker Range Rate RA 

Centerline Displ, Yes 

Lights Up Yes** 

Lights Down Yes** 

Lights Fore Yes** 

Lights Aft Yes** 

Altitude Error RA 

Air Drop Cross Track Error No 

Position Error Yes 

Range from Lead RA 

Bearing from Lead RA 

A Altitude from Lead RA 
Actual Air Release Pt, Yes 
Formation Range RA 

Range Rate RA 

Bearing RA 

Ground Attack TGT, Slant Range RA 

Aim Point Error Yes 

Bomb Fall Line Yes 

Flight Path RA 

Spacing RA 

Dart Firing Range RA 

Azimouth RA 

Elevation Yes 

Air Combat TGT, Range RA 

TGT, Range Rate RA 

TGT, Aspect Angle RA 

TGT, HDG Cross Angle No 
Elevation RA 

Space Path No 



* RA=Reduced Accuracy 

**Cbtainable with Viaeo/Photo System, 
tut not easily otherwise. 
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collection. They tried to fill these gaps by cross-checking, 
where possible, with ether aircraft training. 



The following are observations of the degree of 
commonality of training phases for the six USAF aircraft and 
HAF’s aircraft also: 

(1) TEANSITION (TE) . Transition is a phase of 
combat-crew training. Transition maneuvers appear in a 
common fashion, but the manner in which they are performed 
is significantly different for different aircraft. This 
phase has been already adapted by HAF. 

(2) INSTEOMENTS (INST) . Instruments is a common 
flight phase for all aircraft (also adapted by HAF) and 
conseguently common measurement (criteria) is conceptually 
possible. 

(3) FOEMATION (POEM) . Formation flight is performed 

by all six aircraft as a means to optimally employ composite 
flight and provide individ ual-ship effectiveness. It is 
considered as a common flight phase among all aircraft, 
however, a number of types of formation exist for 
specialized missions, each appropriate only to specific 
aircraft. The formation phase is widely used in HAF by high 
performance aircraft (as in F-4E, A-7D, F-1C, F-5A/B) and is 
very limited among multi-engine aircraft (C-130, DC-6, 

Albatros). Measurement differences will occur between the 
categories of high performance aircraft and multi-engine 
heavy aircraft. 

(4) AIE-AIE INTEECEPT (AA) . Air-to-air intercept 
and weapons delivery utilizing airborne radar is accom- 
plished with only the F-4 and F-106 aircraft (in HAF with 
F-4E and F-1C). While the F-4, F-1C and F-106 maneuvers and 
equipment differ, the same basic measurement requirements 
are presented. Almost the same basic measurement require- 
ments could also apply in F-5A/B of HAF, against the event 
that air-air intercept is accomplished by ground radar. 
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(5) BASIC FLIGHT MANEUVERS (BFH) AND AIR CCHEAT 
MANEUVERS (ACM) . BFM and ACM are grouped together in F-4 
training, and could be grouped also in F-5A/3 and F- 1C of 
HAF, while BFM and Formation are grouped together in A-7 
training. The BFM/ACM grouping was maintained for purposes 
of measurement problem. 

(6) AIR REFUELING (AR) . Air refueling can cccur 

with four of t'he six OSAF aircraft (B-52, F-106, F-4, A-7), 

but is only considered a difficult maneuver for the E-52. 
In HAF air refueling can occur only in F-4E and A-7 but is 
not used since there are not any tankers available. 

(7) GROUND ATTACK (GA) AND DROP. A number of 
training phases are devoted to F-4 and A-7 (both aircraft 
are available by HAF) ground attack (Day, Tactical, Night, 
with various weapons and delivery modes) , but common 
measurement was judged to be appropriate. 

Air Drop (ccmbat airlift mission) maneuvers for 
C-141 and C-130 training perhaps superficially resemble 
level-flight ground attack, but quite different measurements 
are posed. Ground attack and drop phases have been also 
adapted by HAF but in this casethe measurement is different. 

(8) RADAR NAVIGATION AND BOMBING (RNB) . Navigation 
by use of radar, and subsequent delivery on target of either 
ordnance or cargo, occurs with most of the aircraft of the 
sample except the F-106, and compatible mission performance 
measurement is indicated. In HAF this phase is used by all 
aircraft except these that are not equipped with radar 
(e.g., F-5A/B) . 

2- Summar y of Tr a ining Phas es. 

The examination of the USAF study shows that 

measurement (criteria) , for the training phases of the 
sample of six USAF aircraft, are composed as follows : 

Transition 

Instruments 
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Format! on 

Air-to-air Intercept 
Air Combat 
Air Refueling 
Ground Attack 
Air Drop 

Radar Navigation and Bombing 
» All of these training phases (except air refueling) 
are widely used by the aircraft of HAF. From both studies, 
DSAF*s and present, it is concluded that not all of the 
training phases are applicable to all types of aircraft. For 
this reason, it is required to ignore the measurement 
developed in cases where a specific phase could not be 
applied. 

3 . Common Measurement for Maneuv ers 

Each phase of flight, tentatively considered to 
permit common measurement , was examined for detailed measure- 
ment requirements. All needed mesurements, for each maneuver 
used in the USAF study, were provided by (1) interview notes 
with Instructors and Training Managers, (2) Aircraft 
Techical Orders, Dash-one flight manuals for each aircraft, 
(3) Phase Manuals, (4) Instructor Guides, and (5) other 
specialized documents. Important information, as judging 
factors and common errors were noted for each maneuver, 
along with other important remarks that an instructor pilot, 
or training manager, would consider and thus translated into 
objective measurement. 

For example, measurement requirements were noted for 
each aircraft during takeoff and climb-out maneuvers (Runway 
Roll, Rotation, Liftoff, Gear-up, Flaps-up, Climb and 
level-off). A matrix of measurement requirements was thus 
produced, allowing comparison ^across aircraft. It was noted 
that takeoff and climh-out are essentially the same for all 



38 



aircraft, with the exception of the importance of some 
maneuvers (e.g., rotation is critical with the F-106), vari- 
ations in aircraft design (e.g., F-106 has no flaps, B-52 
involves a complicated flap schedule) , and climb profiles 
depend on the specific mission and clearance. 

Although there were some differences, in measurement 
requirements among different types of aircraft, the 
similarities proved to be a lot more. Also, a modular 
approach to measurement was suggested, that is, measurement 
could be produced by examining every maneuver in sequence 
for every type of aircraft. 

All of the above mentioned maneuvers are used by the 
Hellenic aircraft and thus common measurement for maneuvers 
in a modular approach could also be considered by all 
aircraft. 

B. PHOTOTYPE MEASOREBENT. 

After the examination of measurement commonality of 
training phases and maneuvers, the USAF study developed, in 
the form of formatted measurement outputs, examples of the 
required information for training. That is, if a measure of 
centerline deviation was indicated to be desirable, this 
would be noted. In this way they recorded all of the known 
information requirements for a given phase of flight, and 
then, they formed the data found into a format to resemble 
measurement output. This output is termed here as Prototype 
Measurement. 

An example of prototype measurement is displayed in 
Figure 4.1 for takeoff and climb. Details and explanations 
of this (prototype measurement) example is presented in 
Appendix A. A blank or x in Figure 4.1 indicates one or 
more numerical entries to be determined as a result of 
measurement. For every one of the training phases similar 
measurement was developed. 
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In the USAF study the prototype measurement was produced 
for the following trainig phases: 

(1) TRANSITION. For convenience, transition was divided 
into (a) Takeoff and Climb, and (b) Pattern, Land or 
Go-arcund. As previously noted, measurement appropriate to 
the composite of six aircraft is indicated; the F-106 and 
B-52 require special treatment. 

(2) INSTROMENTS. Measurement for instrument flight is 
treated as the sum of (a) basic aircraft control 
performance, and (b) navigation performance with respect to 
air traffic control requirements. While required measure- 
ment modules for each of these classes of measures can be 
specified, it is difficult to present detailed measurement 
except for specific clearances and published procedures. 

(3) FORMATION. Measurement for formation flight is 
super-imposed onto mission performance measurement, thus 
making formation measurement difficult to isolate. In 
particular, tactical formation performance was not clearly 
identified in this study in an objective quantitative 
fashion. It may be necessary to rely heavily on instructor 
subjective measurement (which may be quite satisfactory when 

* the instructor is in a position to observe performance ) , 
except for measurement associated with join-up, close forma- 
tion, and in-trail formation. 

(4) INTERCEPT. In order to be specific, intercept 
measurement was based primarily on the F-106, however, the 
intercept problem is essentially the same as the F-4. Of 
cource, a two-man crew performance is expected to be better 
in the F-4. There is a radar observer to perform the scope 
work and differences in the equipment and capability suggest 
that slightly different strategies might be employed. 
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(5) AIF. COMBAT MANEUVERS, Prototype air combat maneu- 
vering measurement dealt primarily with set-ups during 
initial training and for dart^ firing. 

(a) Air combat set-ups involve placing attacking and 
defending aircraft in fixed initial positions, then freeing 
them to perform a maneuver, and subsequently judging from 
the final position whether the maneuver was properly 
performed and whether proper advantage of the tactical situ- 
ation was taken. Thus, measurement can be directed to 
description of the maneuvers performed (e.g., hard turn, 
hi-lo-speed yo-yo, scissors, barrel roll, etc.), and to 
determining whether a given student was able to improve his 
situation. 

(b) For dart firing the prototype measurement assumes a 
butterfly pattern or the equivalent. A pass is made over the 
target, a time hack is taken crossing the dart, the pilot 
must circle back to make an intercept to put a hole within 
the target in a given amount of time. Thus, the time and 
hits on each pass is measured; additionally the range, 
azimuth, and elevation at the beginning and end of firing 
describe the firing position. Fouls are called for low 
airspeed and for firing within a minimum firing range. 

(6) AIR REFUELING. As Air refueling is especially diffi- 
cult in E-52 combat-crew training, the prototype measurement 
was tailored to the E-52 tasks and to Strategic Air Command 
requirements. 

(7) GROUND ATTACK. During training, ground attack is 
divided into ground attack, ground attack night, ground 
attack tactical. Ground attack measurement was dictated by 
information needed for standard error analysis of weapons 
delivery accuracy and by ground attack procedures. Some of 



3The dart is a kind of air to air target, towed by a tow 
aircraft. 
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the measures are specially designed to apply to the A-7D 
heads-up display. 

(8) AIR DROP. Extensive prototype measurement was 
dictated for the Comtat Airlift Mission since very detailed 
procedures are adhered to throughout the many portions of 
the mission. Crew performance is especially important in 
this mission. 

(9) RADAR NAVIGATION AND BOMBING. Prototype measurement^ 
for Radar Navigation and Bombing is heavily dependent on the 
characteristics of the avionics used. As only the B-52 is 
equipped with low-level terrain avoidance radar (of the 
aircraft sampled in this study), measurement for these 
maneuvers has been tailored to this application. 

Of course, from the examination of the prototype 
measurement we can see that the measurement requirements are 
very extensive and complex. For example, it may be seen that 
to describe just Takeoff and Climb, it is necessary to 
compute 50-100 numbers. Subsequently, if full mission 
measurement is required, including transition, instruments, 
formation, and weapons delivery, it will require a very 
large set of descriptive numbers. But probably, this detail 
is very necessary to support the training process. The 
instructor may need considerable detail to perform his job 
well. As a consequence, the prototype measurement could be 
very useful for HAF, since today the evaluation of aircrew 
proficiency in skills associated with advanced flying 
training is primarily based upon the subjective judgement of 
instructor pilots. 

C. MEASOREMENT PARAMETERS. 

Measurement is the process of producing measures which 
are indices of performance such as the conditions existing 
at the time of weapon release during ground attack. 



42 



deviations from the clearance during IFR flight, and flight 
conditions at lift off. The measures are commonly computed 
from flight variables (e.g. , altitude, airspeed, heading, 
etc.) and other raw information which must be recorded at 
some time; these are termed measurement parameters. The 
measurement parameters must be specified so that the 
required sensors and recording equipment can be determined. 
However, since the measures are the result of a computation, 
the details of the computation must be known so that the 
inputs to the computation (the parameters) car be 
established. 

Figure 5.1 depicts the relationship between the speci- 
fied measures, the ccmputation , and the measurement parame- 
ters. The corresponding data processing is shown in Figure 
5.2. It is assumed that the flight maneuvers will be 
divided into segments, so that different measures may be 
computed as appropriate for each segment (for example, 
different measures are required during takeoff roll than 
during climb- out) . Consequently, it may be seen that the 
method of determining when to start and stop the computation 
of a specific measure may require the recording of measure- 
ment parameters in addition to those required during measure 
computation (e.g., the recording of weight-off-the- wheels to 
indicate liftoff). 
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Figure 5.1 Identification of Measurement Parameters 




Figure 5.2 Example Raw Data Processing 
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MANEUVER MEASURE FUNCTION CONDITIONS COMMENTS 
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Figure 5,3 Example Measurement Specif ication. 



In Figure 5.1, the output measures (0) correspond to the 
information requirements symbolized by the blanks in the 
prototype measurement forms; that is, if the prototype 
measurement forms indicated that a measure of centerline 
deviation is needed, then parameters must be recorded and 
computations developed which will answer the required 
measure, the following types of parameters may be needed in 
addition to the basic test parameters (M) (for example just 
mentioned, the basic test parameter would be a recording of 
the deviation from the runway centerline during takeoff 
roll) : (1) parameters for implementing logic to start and 
stop measurement computations (S) , (2) information related 
to desired performance (D) ,and (3) error information derived 
from the differences between actual and desired performance 
(E) . In short, given output measures (0) , to determine 
other parameters which must be sensed (M,S,D,E), it is 
necessary to determine logic and computations to be used in 
measurement data processing (i.e., the measurement 
algorithms) . 

Assuming automated measurement, i.e., parameters are 
automatically recorded for subsequent computer analysis, the 
primary details of measurement computation (Figure 5.2 shows 
a representative flow diagram) are presented in Figure 5.3 
for each maneuver and maneuver segment of combat-crew 
training phases. The figure 5.3 indicates the name of each 
measure, the specific function to be computed, and the 
start/stop conditions for controlling computation. For 
example, centerline deviation during the takeoff roll is 
desired output information, the average, minimum and maximum 
deviation are the specific computations which should be 
performed between brake release and rotation. Comments are 
also provided as considered appropriate by the analyst to 
point up alternatives, or where problems may be encountered 
during design. 
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since figure 5.3 indicates the functions to be computed, 
the conditions under which computation should occur, and 
indirectly the source information upon which to base compu- 
tation, the basic information is provided to allow prepara- 
tion of computer programs for automatic measurement. 

When a list of parameters was developed to show all the 
required sensors and recording needed for total measurement, 
it was seen that the resulting parameters could be placed in 
the following overlapping categories: (1) pictorial infor- 
mation (e.g., out- the-window radar), (2) analog information 
(e. g., time-varying quantitative, such as airspeed), 
(3) discrete information (e.g., weapon release), (4) audio 
information (e.g., communications) , and (5) desired 
performance and existing conditions. These parameters were 
later assigned to alternative devices for data acquisition. 
After tradeoff analyses were conducted, a hybrid audio- 
video/photo -digital recording system was adopted. Although 
many parameters could be acquired by either video-photo or 
digital recording devices (to allow a partial system to have 
a stand-alone capability) , tentative parameter allocations 
are listed in Tables 4 and 5. Audio recording will be 
accomplished with either video-photo or digital recording 
devices. Desired performance and conditions are manually 
derived from briefing/debriefing sessions and documents. 
Additionally, spatial coordinates (X-Y data) may be obtained 
if data are collected on an instrumented range including 
tracXing-radar equipment, although equivalent information 
may be available from video-photo recording as shown in 
Table 3. 

After detailed trade-off analyses, the use of video or 
photo techniques was emphasized due to lower costs, flexi- 
bility of application and simpler development compared to 
other all-electronic techniques; however, the problems asso- 
ciated with cockpit installation and an unfortunate tendency 
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TABLE 4 

Parameters for Digital Data Acquisition 



1 



T 

R 

A 

N 

S 

I 

T 

I 

0 

N 



IFIAAGDA A 
NONIIRAI C 
SRTRRORR C 
T M E U T U 

RARRDN C R 
UTCERDFO A 
MIEFO IMC 
EOPUPARB Y 
N N T E TEA 

T L T T 

S A 

C 
K 



AIRCRAFT PARAMETERS 
T. Pltch“1[7i tch Rate) 

2. Roll 

3. Heading 

4. Airspeed 

5. MACH 

6. Altitude 

7. Vert. Vel. 

8. Angle of Attack 
9- Accelaration (G) , 

10. Power (RPM, EPR, 
TIT, Fuel, Flow) 

11. Fuel Quantity 



X X 
X X 
X X 
X X 
X 

X X 
X X 
X X 
X 

X X 



X X X X X 

X X X X X 

XX X X X X 

X X X X X X 
X X 

X X X X X X 

X X X X X X 

X XX 

X XX 

X X X X X X 

XX X 



CCNTROL PARAMETERS 

“l“R^clc~7PrTcIir~ 

2. stick (Roll) 

3. Rudder 

4. Flap Position x 

5. Star Trim Position x 



X X 

X X 

X 

X X 



X 

X 

X 

X 



±1 degree 
±1 degree 
±1 degree 
±1 knot 
±.02 MACH 
±10 feet 
±50 fpm 
±1 unit 
±. 5 G 

±1% Full Scale 
±5% Full Scale 



±5% Full Scale 
±5% Full Scale 
±5% Full Scale 
±5% Full Scale 
±5% Full Scale 



BINARY 

~1 



DISCRETE 
TEr u^ 



P ARAMETERS 

Reverse x 



Brakes 
Nose Gear 



2 . S p ee d 
3,4. Main, 

Contact 

5. Ncse Steer Engaged 

6. Gear Select 

7. Drag Chute 

8. Nteel Brakes 
9.10. Red, Green light 
11. Weapon Release 
(Pickle) 

12,16. Crewmember 
Voice Switch 
17,19. Marker Beaccn 
20,24. Event Marker 



X X 

X 

X 

X 

X 

X 



X 

X 

X 



X 



X 

XXX 

XXX 



X 

X 

X 

XXX 
X X X X 
X X X X 



TIME 

1. GMT (Range Time) xxxxxxxxx Hrs, Min, Sec, 

1/100 Sec. 
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for loss of data should be noted, but it is believed neces- 
sary to accept these deficiencies for low-cost combat-crew 
training application. 

Accuracy reguireuents for measurement parameter sensing 
and recording are listed at the right-hand side of Table 4. 
These accuracies are referenced to the information displayed 
to the crew (for example, the required airspeed tolerance is 
plus or minus 1% of the deviation between recorded values 
and those displayed to the pilot on his airspeed indicator) 
since the criterion given the crew is that they maintain 
vehicle parameters within specified tolerances referenced to 
their instruments (typically 5-10 knots for airspeed 
control) . The tolerances listed in Table 4 are approxi- 
mately 1/10th the tolerances required of the crew. 

As a result, (a) if HAF has to make a choice between 
video/photo and digital recording approaches to measurement 
system design, then the video/photo recording would be 
chosen for cost, information provided, flexibility and ease 
of use. (b) A hybrid system combining the advantages of 
both is preferable to either type of recording alone. Due 
primarily to cost, the bulk of measurement parameters would 
he derived from a video/photo system, and the remainder with 
a small digital recording capability. 

D. HEASDBEMENT DESCBIPTIONS 

The gross operations involved in measurement computation 
are presented in the flow diagram in Figure 5.2 Each 
parameter must be sampled (at a sampling rate of 2, 10 or 20 
times a second, depending on the application) and tested to 
determine if conditions are appropriate to start measurement 
computation, and later to stop computaton. During the 
measurement interval (or at specific conditions, €. g. , 
flaps-up), one of the following statistics is calculated: 
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TABLE 5 

Parameters for Video/Photo Data Acquisition 



PHASE 


PARAMETERS 


Transition 


Runway Centerline Deviation, Lateral 
Drift, Threshold Crossing, Distance Down 
Runway, Ground Track. 


Instruments 


TACAN (Frequ., Course Set, Course Error, 
Bearing, DME) : 

VOE {Frequ., Course Set, Course Error, 
Bearing) ; 

ILS (Frequ., Localizer Error, Glide Slope 
Error, Marker Beacon) . 


Intercept 


Target use of ECM, Maneuvering 
Radar: Azimuth. Elevation, Range, Range 
Rate, Range Gate, Steering Dot Error, 
Firing Circle Radius, Lockon Pulse. 

IF Gain, Video Gain, Erase Intensity. 


Refueling 


Range to Tanker. Range Rate, Probe 
Engagement, Centerline Displacement, 
Lights (Dp, Down, Fore, Aft) , Attitude 
Error. 


Air Drop 


Crcsstrack Error, Groundspeed, Terrain 
Clearance, Range/Bearing/ AAltitude from 
Lead A/C, Eed/Green Drop Lights, Actual 
Air Release Point. 


Formation 


Spacing: Range, Range Rate, Bearing, 
A A Ititude. 


Ground Attack 


Target Slant Range, Aim Point Error, Bomb 
Fall Line, Flight Path Error, Spacing 
in Range Pattern. 


Dart Firing 


Range, Azimuth, Elevation, Hits. 


Air Combat 
Maneuvers 


Target Range, Range Rate, Aspect Angle, 
Heading Crossing Angle, Elevation. 
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(1) the value of a parameter^ (2) minimum value^ (3) maximum 
value, (4) mean, or (5) standard deviation. Thus, measure- 
ment is defined by specifying start/stop conditions and one 
of the five statistics. 

An example of a measurement specif ication patterned in 
this fashion is shown in Figure 5.3. Measurement specifica- 
tions were produced for each common training phase. 

While these specifications initially assume automatic 
recording and computation, they can also be used to describe 
manual data processing (e.g., a discrete signal indicating 
wheels-up) . These parameters can also be used to start and 
stop manual processing such as scanning for out-of-tolerance 
conditions. During the measurement interval, the value, 
minimum or maximum cf a parameter may be determined manu- 
ally, but manual processing for computation of a mean or 
standard deviation is judged to be excessively laborious and 
time-consuming since a large number of data samples (at 2 or 
more times a second) is necessary. Consequently, the 
measurement specifications, as exemplified in Figure 5.3, 
are suitable for (1) defining software for digital computer 
measurement processing, or (2) manual processing procedures. 

Today, HAF specifies its measurement specif ications 
mainly by manual processing procedures. Lately, HAF has been 
interested in computer measurement processing. Thus, HAF 
should consider the measurement specifica tions mentioned in 
this section. 

E. CCHMONICATION ANALYSIS 

Measurement for crew performance overlays the system 
performance measurement thus far discussed- The performance 
demonstrated in A-7 or F-106 aircraft clearly involves only 
one man; however, the same mission may be flown by two men 
in the F-4 aircraft, requiring additional measurement to 



52 



investigate crew interaction and diagnose individual 
performance contributions. The combat airlift mission in 
the C-130 and C-141 aircraft involves such close coordina- 
tion of pilots, navigators, loadmasters and engineers that 
it is difficult to isolate an individual's performance (even 
the crewmembers themselves cannot be sure of the adequacy of 
their performance) . 

An individual's performance can be assessed by relating 
measurement at his workstation to overall system 
performance; this type of measurement is subsumed under the 
previous measurement discussion. However, the interaction 

between crewmembers requires analysis of communications 

presenting somewhat different requirements than system 
performance measurement. The following paragraphs present 
important topics related to crew performance measurement. 

1 . Communicatio n M easurem e nt Categories. 

Communications measurement must treat at least two 
gross types of crew interaction; (1) information is 
exchanged to aid another crewmember in performing his duties 
(e. g., when the F-4 Weapon System Officer acts as a good 
"copilot") , and (2) a crewmember provides a directive role 
in guiding another's performance (e.g., when the F-4 Weapon 
System Officer provides directive commentary to the Aircraft 
Commander in air-air intercept) . In the later case direct 
links between auditory commands and system performance can 
be identified, allowing communication to be measured in 
terms of resulting performance changes. 

Six categories of measurement related to information 
transfer were examined; 

( 1 ) Timing . Measures of information timing should 
relate to (a) jammimg more important messages, (b) providing 
information at the wrong time, (c) delay in providing 
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information, and (d) providing information at a rate not 
permitting effective response by another crewmember. 

(2) A ccur a cy . Measures of accuracy require 
comparison of what is said in relation to the measured situ- 
ation (e.g., was altitute reported correctly?). 

(3) Brevity . As radio and interphone traffic often 
exceed channel capacity in combat, measurement should 
address communication duration and comparison brevity code. 

(4) Number and Freque ncy. Also in relation to 
communication brevity, the number of communications and 
frequency of communications can be measured. 

(5) I nforma tion con tent . Measures of information 
transmitted per unit time are quite important although 
usually practically difficult to obtain. Other measures such 
as time, number and frequency are often confounded since a 
crewmember may convey much information in a short time or 
few transmissions, while another may say little in a long 
time or many transmissions; without knowledge of the infor- 
mation content it would be difficult to evaluate these 
situations. 

(6) Per for ma n ce chan ges. The performance changes of 
the vehicle, desired as a result of comm unicating, define 
measurement in terms of links between auditory data and 
system/mission performance data. For example, turning 
performance can be measured following a "hard-as-possible" 
direction to the pilot of an F-4 aircraft. 

2. Au d itory D ata Proce s sing . 

A compu ter-assisted manual auditory processing 
system is required since automatic voice decoding equipment 
is not available. Expert personnel are therefore required to 
identify complex performance and to structure processing 
rules for data clerks who reduce data to a form allowing 
input into a computer. 
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Three auditory data processing problems imposed 
requirements for measurement system design: 

a. Identification. 

In spite of normal communications clutter, it is necessary 
to clearly identify who is talking, even if two transmis- 
sions are simultaneously made (jamming) . Voice key 
circuitry is recommended to provide a digital signal indi- 
cating which crewmenber is talking. 

b. Synchronization. 

Audio data must be synchronized with other data 
recording to permit relating auditory information to corre- 
sponding performance changes. Each recording device must 
therefore include an audio voice track. 

c. Data Reduction. 

Manual functions in audio data processing are 
unavoidable, but can be minimized through computer- 
assistance. Audio data playback must be accompanied by 
display cf performance parameters, especially the parameter 
of TIME, to allow computer correlation of manual audio-data 
entry with the digital data base. A convenient means for 
manual data entry is a desirable feature. 

These data processing problems must be 
considered in the design of a combat-crew training measure- 
ment system. Thus, since HAF has at most the same aircraft 
as the aircraft of the OSAP bases from which data were 
gathered, HAF must apply the same concepts for the purpose 
of deriving combat crew performance measurement. This 
approach will solve its problem of continuously optimizing 
the combat readiness cf its combat pilots. These concepts 
(measurements) that were discussed in this chapter are the 
following: 
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(1) Measurement Conconality 

(2) Common Measurement 

(3) Prototype Measurement 

(4) Measurement Parameters 

(5) Measurement Descriptions 

(6) Communication Analysis 

Of course, the value of experience in combat is 
not easily determined. History says, experienced fighter 
pilots have done better than inexperienced pilots in combat 
situations. Intuitive judgement says that the greater the 
experience level, the greater the chance a unit has for 
success. 

Modern fighters are only as effective as the 
pilots who fly them. The HAF should continuously evaluate 
the need for experienced fighter pilots in the combat ready 
units. The price may be high, but the price of defeat in the 
next battle for the air may be even greater. At the end of 
WWI (World War I) a doctrine was written, ”... if you hold 
the air you cannot be beaten, if you lose the air you cannot 
win.” [Bef. 28]. 

So far, in order to establish criteria for 
combat readiness of HAF*s combat pilots, we have dealt in 
chapter II with the principles of human performance, in 
chapter III with the criteria generally and their measure- 
ment, in chapter IV with the introduction of the combat- 
ready crew performance measurement, and finally in chapter V 
we derived the combat-crew performance measurement 
techniques. 
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VI. COHC IDSI ONS OD RECOMMPAII^S 



The Hellenic Air Force has always been interested in 
keeping the highest level of performance for its combat 
pilots and keep up with the latest of technology achieve- 
ments. Studies^ as that of DSAF, have always been of a great 
concern, for the purpose that new techniques and theories 
can be revealed in its benefit. In this way, HAF keeps up 
with the latest and increases the efficiency and effective- 
ness of its combat pilots. The necessity of improving 
training performance information of its pilots requires new 
methods of measuring flight crew performance during combat 
crew training. As we have seen in this study, criteria for 
combat readiness of fighter pilots are ; 

(1) Parametric referent or standard of performance. 

(2) Parametric limit about the parametric standard. 

(3) System component criteria. 

(4) Test criteria. 

(5) Multidimensional in nature and represent the 
complefe desired end result of a system. 

From the examination of this and the USAF study we can 
conlude that: 

(1) Opening a new flow of information can have a major 
influence on training technology. The execution of effective 
contemporary training, the development of new training 
devices, and the exploitation of powerful concepts such as 
adaptive training, and learner-centered instruction, all 
depend on information available through performance 
measurement. 

(2) The role of performance measurement may be that of 
causal input, a catalyst, or a weak link in a system chain, 
but the net effect of better performance measurement in any 
case is a positive and possible revolutionary improvement. 
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(3) The major difficulty that stands in the way of 
improvements through performance measurement is that 
adequate performance measurement systems do not exist where 
needed. The present study has defined a guide for measure- 
ment in a manner permitting adaptation to specific needs and 
budgets. 

(H) The status of combat-crew measurement and other 

areas cf complex man-machine performance is reflected by the 
measurement analysis. The measurement described here 
reflects the critical dimensions of performance, and agrees 
with the measurement structure used by operational training 
personnel. Given this capability a number of future 
advances are possible. First, use of this measurement will 
lead to efficient interim and immediately available measure- 
ment for training. Second, improvements in the generation 
of optimally efficient measurement sets may be expected. 
Third, clarification of the relationship between objective 
and subjective measurement should be possible. None of 
these results is going to occur without an acceptable 
performance measurement system and appropriate 

experimentation. 

This section is ccncluded with the following recommenda- 
tions for future development opportunities of the HAF: 

(I) To consider measurement in the context of combat- 
crew training. 

(2) To assess measurement, that is to identify potential 
measurement indicated by combat-crew training personnel. 

(3) To assess the constraints placed by the environment 
on feasible, usable measurement systems. 

(4) To establish the model of instructional system 
development . 

(5) To establish a measurement system for the opera- 
tional environment. 
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To establish an automated measurement system that would 
relieve the instructor pilot, to a maximum extend, from the 
requirements of having to record a great deal of information 
manually on the basis that such activity degrades his 
ability to competently instruct his student. 

(6) For the purpose of deriving combat-crew 

performance measurement to consider (a) measurement 

commonality, (b) prototype measurement, (c) measurement 

parameters, (d) measurement specifications, (e) communica- 
tion analysis. 

(7) If HAF has tc make a choice between video/photo and 
digital recording approaches to measurement system design, 
then the video/photo recording could be chosen, as a 
performance measurement system, because of its low cost, 
information provided, flexibility and ease of use. A hybrid 
system combining the advantages of both is preferable to 
either type of recording alone. Due primarily to cost, the 
bulk of measurement parameters would be derived from a 
video/photo system, and the remainder with a small digital 
recording capability. 
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APPENDIX A 



PECTOTYPE MEASOEEMENT 

In this Appendix an example of prototype measurement is 
presented for takeoff and climb. The format used is to 
present a discussion together with prototype measurement, 
indicating through the Figure A. 1 the types of information 
which are considered important to a description of pilot 
performance. 

TAKEOFF S CLIMB 

All aircraft takeoff and climb to a cruising altitude 
and configuration. Fixed-wing aircraft perform these maneu- 
vers in hasically the same way; however, at a detailed level 
there are distinct differences between aircraft. Thus, 
measurement must be tailored to each aircraft, but the 
general structure of such measurement may be defined so that 
the essential elements are constant across aircraft. The 
following sequence is rather basic: Takeoff roll. Fetation, 
Liftoff, Gear-up, Flaps-up, Climb and Level-off. The infor- 
mation desired within each of these flight maneuvers may 
also be expressed in a substantially common manner. 

Co nditions . To properly interpret measurements made 
during a particular flight, information on the conditions 
existing at the time are needed. The gross weight, wind 
direction and velocity, runway direction and length, 
temperature, altimeter setting, field elevation, and posi- 
tion of the aircraft in formation, are reference data for 
the evaluation of performance. 

Tak eoff roll. The takeoff will be assumed to begin with 
the application of power. The takeoff roll maneuver will be 
considered finished at rotation. The objective is to 
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TAKEOFF 8 CLIMB* 



CONDITIONS: 

■Cross WY: Ring: / _ Eunway: / 

Temp.: Alt. Set.: ‘FTeld Elev. Form Pos. : 



TA KE O FF BOLL: (To powe r until rot ati on) 

Power Se^:_ _ TentefTine Pev. : Min, Max, Av. 

Eeject SpeeP: Computed Heading:Min, Max, Av. 

Time: Dist: Bank: s Max, L Max 



EOTATICN: (Nose g ear off until £itch att. e sta blished) 

Po^.Tpeed: ”PfaP. Trim: 

Pitch: Bate: Bank:_ 

Final: Centerline Pev. : 

OvershooT?~ Heading: 



LIFTOFF: (Pos. Vert. Vel.) 

Iirspeed: “Pitch: Bank: Hdg: 

Vert. Vel. "After: Sec.: 



GEAE-DP: f Han dle up until qear-up 8 locked) 

“TSear-Pp S^PT” I vr77“XniT:TIII 7.T7"PInal: 
Pitch: Bank: Hdg: 



Trim 



up to full up) ** 

P^ST^OnXy IAS PITCH ALT VV TEIM 

Pitch: Bank: Hdg: Start x x x x x 

A/S jfiPiT) TFINAL) 1st Pos X XXX x 

VV (INIT) (FINAL; 2nd Pos x x x x x 

ALT (INITT“ (FINAL) Full x x x x x 



FLAPS UP: ( Start 

Bank: 



CLIMB 8 LEVEL OFF: (D epe nds on Flight Plan) 





PWB 




Accelerate 


X 


X 


Climb A/S 


(#1) 






(#2) X 


X 


Climb MACH 


X 


X 


Level-off 1 


(Alt-105? VV) 




1 


[to Cruise) x 


X 



MACH 


INIT 

HDG 


FINAL 
ALT ALT 


PITCH 


TRIM 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 



* Also, mandatory communication 8 instances where A/C 
limits are exceeded. 

**F-106 has no flaps 



Figure A. 1 Example of Prototype Measurement. 
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accelerate in a straight line along the centerline, or 
parallel to the centerline, with wings level. Power and 
resultant acceleration must be checked; for heavy aircraft 
and/or short field takeoffs, accelaration checks are 
formally performed. lime and distance along the runway are 
checked against airspeed to determine if necessary accelera- 
tion performance is lacking in the time to safely stop the 
aircraft. Beject speed is noted in case of an emergence. The 
formation flight leader must slightly reduce power to allow 
a margin of thrust control for other members of the flight. 

Ro tation . Proper rotation is normally necessary to 
achieve predicted takeoff performance. Rotation will be 
defined as the activities between the time that the nose 
gear lifts off the runway until the time that a stable pitch 
attitude is established. Stabilizer trim is important, bank 
angle, centerline and heading deviations should be small. 
Rotation should occur within 1-2 KIAS of the desired rota- 
tion speed. The rate of rotation should not be either too 
large or too small. A specific pitch attitude should be 
established witout overshoot or oscillation. 

li ftof f. Liftoff is a dicrete event, occurring when 
vertical velocity is positive. At this time, the airspeed, 
pitch angle, bank angle, and heading are noteworthy. The 
vertical velocity a short time after liftoff may also be 
measured to indicate whether the aircraft is positively 
airborne, or if there is any tendency to settle back to the 
runway. 

Ge ar-u p. Measurement should be taken from the time that 
the gear handle is raised until the time that the landing 
gear are up and locked. The initial speed at which the gear 
are raised, the change in vertical velocity during the time 
that the gear are coming up, and pitch, bank, and heading, 
should be measured. 
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Fla ps-u p. Flaps-up measurement is treated in somewhat 
the same manner as for gear-up, for the tasks are somewhat 
the same: a configuration change is occurring which 

presents a perturbation in longitudinal control. A trim 
change occurs, and pitch, bank, and heading must be 

controlled. Normally, flaps must not be raised before a 
specific altitude and airspeed (but before maximum flaps 
speed), and during the transition to flaps-up, changes in 
airspeed, vertical velocity, and altitude indicate whether 
the maneuver is properly performed. 

The B-52 presents a special measurement requirement 
since a specific speed schedule must be maintained as flaps 
are raised; in addition to airspeed, pitch angle, altitude, 
vertical velocity, and stabilizer trim are of interest 
during this period of time. 



Cli mb and Level-off . For each aircraft, there are a 
number of methods for climb-out depending on the flight 
plan, and desires for economy or performance. It may be 
desirable to measure climb performance from liftoff, or to 
start when the aircraft is in a clean configuration. This 
phase may be divided into the following parts: acceleration, 
maintain climb airspeed (may be several increases in 
airspeed during the climb) , maintain climb Mach number, and 
level-off (normally level-off begins at an altitude which is 
below cruise altitude by ^0% of the vertical velocity) . 
Power, airspeed, Mach, heading, initial and final altitude, 
pitch angle, and trim, are parameters which may be measured 
during each portion of climbout. 
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